Timo Sirainen [Fri, 1 May 2026 16:56:21 +0000 (16:56 +0000)]
[PATCH 10/12] lib-master: Move config_version_find() from src/config/config-parser.c
Rename to dovecot_config_version_find() and expose it via
master-service-settings.h so non-config callers (e.g. the upcoming
dovecot_storage_version validation in master_service_settings_check())
can share the same supported-versions list. No behaviour change.
Gbp-Pq: Name 0010-lib-master-Move-config_version_find-from-src-config-.patch
Timo Sirainen [Thu, 30 Apr 2026 12:11:57 +0000 (12:11 +0000)]
[PATCH 07/12] lib-index: Add keyed xxh64 strmap format v2, gated by config version
The on-disk hash stored in the strmap file is now optionally a keyed
xxh64_to_32() with a per-file random 64-bit IV stored in the file
header. This replaces the previous plain crc32_str_nonzero() output
and hardens the strmap against deliberately-collided message-id
hashes. The v1 (crc32) format remains fully readable and writable
so older configurations keep working without forced rebuilds.
On-disk:
- v1 header is 8 bytes: version, 3 unused, uid_validity (unchanged).
- v2 header is 16 bytes: version, compat_flags, 2 unused, uid_validity,
hash_iv. The first 8 bytes of v2 align with v1 byte-for-byte, so
the open path reads 8 bytes, dispatches on version, and reads the
trailing 8 bytes only for v2.
Runtime:
- strmap->enable_xxh64 selects the format used when creating a new
file (or recreating one whose uid_validity has changed).
- view->format_version follows the on-disk file when one exists; for
fresh files it follows the strmap-wide preference.
- Renumber recreate_write() preserves view->format_version, so we
never silently migrate an existing file across the threshold while
it is still in use - the migration only happens at open time.
- When enable_xxh64 is TRUE and the on-disk file is v1, open treats
it as a version mismatch: the file is unlinked and the next sync's
recreate_write() produces a v2 file. Below the threshold v1 stays
v1 indefinitely.
- strmap_hash_str() dispatches per-view, so v1 files keep using
crc32 hashes and v2 files use keyed xxh64 even in mixed setups.
Gbp-Pq: Name 0007-lib-index-Add-keyed-xxh64-strmap-format-v2-gated-by-.patch
Timo Sirainen [Mon, 13 Apr 2026 10:52:15 +0000 (12:52 +0200)]
[PATCH 06/12] lib-index: Rename crc32 variables/parameters to hash in strmap
Pure rename - no functional change. The stored 32-bit value is still
produced by crc32_str_nonzero(); only the identifiers change to prepare
for swapping the hash algorithm in the next commit.
Timo Sirainen [Mon, 13 Apr 2026 10:38:50 +0000 (12:38 +0200)]
[PATCH 04/12] lib: Key str_hash/strcase_hash with a random seed to prevent HashDoS
hash_init() fills a process-wide uint64_t hash_iv via random_fill()
at lib_init() time (after random_init()). str_hash() and strcase_hash()
pass hash_iv as the xxh64 seed so an attacker cannot predict bucket
placement and manufacture collision chains.
Gbp-Pq: Name 0004-lib-Key-str_hash-strcase_hash-with-a-random-seed-to-.patch
Timo Sirainen [Mon, 13 Apr 2026 10:33:51 +0000 (12:33 +0200)]
[PATCH 02/12] lib: Replace str_hash/strcase_hash with xxh64
Replaces the old ASU-derived shift-and-XOR hash with xxh64_to_32(),
which provides better distribution and avalanche properties.
strcase_hash feeds each i_toupper()'d byte through the streaming API.
Gbp-Pq: Name 0002-lib-Replace-str_hash-strcase_hash-with-xxh64.patch
Timo Sirainen [Mon, 13 Apr 2026 10:30:03 +0000 (12:30 +0200)]
[PATCH 01/12] lib: Add XXH64 hash implementation
Adds a streaming XXH64 (non-cryptographic) hash with init/loop/result
API, a one-shot xxh64_data(), and an xxh64_to_32() XOR-fold inline.
Registered in hash_methods[] for use via the generic hash_method API.
Gbp-Pq: Name 0001-lib-Add-XXH64-hash-implementation.patch
Timo Sirainen [Wed, 22 Apr 2026 09:58:56 +0000 (12:58 +0300)]
[PATCH 2/2] imap-hibernate: Use imap-parser API for parsing DONE command
Replace the ad-hoc DONE/IDLE tokenizer with imap-parser.h: use
imap_parser_read_tag() to read the tag and imap_parser_read_word() to
read the DONE and IDLE keywords. This delegates bounds checks and
character validation to the shared parser.
Gbp-Pq: Name 0002-imap-hibernate-Use-imap-parser-API-for-parsing-DONE-.patch
Timo Sirainen [Thu, 16 Apr 2026 21:12:13 +0000 (23:12 +0200)]
[PATCH 1/2] imap-hibernate: Fix out-of-bounds read when parsing DONE command tag
The tag-skipping loop was missing a size>0 guard, so a malformed
DONE command with no space/CR/tab terminator after the tag would
read one byte past the end of the buffer. Also add a \0 check to
stop on embedded null bytes, which are not valid tag characters.
Gbp-Pq: Name 0001-imap-hibernate-Fix-out-of-bounds-read-when-parsing-D.patch
Aki Tuomi [Mon, 25 May 2026 10:38:13 +0000 (10:38 +0000)]
[PATCH 2/2] managesieve-login: client_skip_line() - Discard data when newline is not found
Without this, data without a newline (e.g., from a lone CR protocol violation
detected by the parser) stays in the buffer indefinitely. This prevents the
server from ever detecting client disconnect, causing a deadlock where both
sides wait for the other.
The post-login server-side client_skip_line() in managesieve-client.c already
handles this correctly by always calling i_stream_skip().
Gbp-Pq: Name 0002-managesieve-login-client_skip_line-Discard-data-when.patch
Timo Sirainen [Fri, 17 Apr 2026 13:15:54 +0000 (13:15 +0000)]
[PATCH 3/3] lib-storage: thread - Limit ancestor chain traversal depth to prevent O(N^2) CPU usage
The per-message References limit (MAIL_THREAD_REFERENCES_MAX) prevents a
single crafted message from causing O(N^2) traversals in
thread_node_has_ancestor(). However, multiple crafted messages each
containing 1000 References entries can build an arbitrarily deep ancestor
chain across the mailbox, causing the same quadratic blowup spread over
many messages: processing email k costs O(k * MAIL_THREAD_REFERENCES_MAX)
steps, giving O(M^2 * MAIL_THREAD_REFERENCES_MAX) total for M emails.
Fix this by limiting the traversal depth in thread_node_has_ancestor() to
MAIL_THREAD_REFERENCES_MAX steps. When the limit is reached the link is
dropped, bounding per-email work to O(MAIL_THREAD_REFERENCES_MAX^2)
regardless of how deep the chain was built by prior messages.
Gbp-Pq: Name 0003-lib-storage-thread-Limit-ancestor-chain-traversal-de.patch
Timo Sirainen [Thu, 16 Apr 2026 16:26:47 +0000 (16:26 +0000)]
[PATCH 2/3] lib-storage: thread - Limit References: header msgid count to prevent O(N^2) CPU usage
Cap per-message References ingestion at MAIL_THREAD_REFERENCES_MAX (1000)
in mail_thread_map_add_mail(). Without this limit a single crafted email
with N unique Message-IDs triggers N(N-1)/2 ancestor traversals in
thread_node_has_ancestor(), allowing unauthenticated DoS via mail delivery.
Gbp-Pq: Name 0002-lib-storage-thread-Limit-References-header-msgid-cou.patch
Timo Sirainen [Fri, 17 Apr 2026 13:57:24 +0000 (15:57 +0200)]
[PATCH 14/14] lib-mail: Limit total MIME parameter count per message to 200 000
Content-Type and Content-Disposition parameters are both parsed through
parse_mime_parameters(). A message with many MIME parts each having many
parameters can accumulate millions of struct message_part_param entries.
Add remaining_mime_params to struct message_part_data_limits (combined
budget for both Content-Type and Content-Disposition), initialised to
MESSAGE_PART_DATA_MAX_TOTAL_MIME_PARAMS (200 000). Cap params_count to the
remaining budget before allocating and deduct accordingly.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Gbp-Pq: Name 0014-lib-mail-Limit-total-MIME-parameter-count-per-messag.patch
Timo Sirainen [Fri, 17 Apr 2026 13:56:37 +0000 (15:56 +0200)]
[PATCH 13/14] lib-mail: Limit total Content-Language tag count per message to 100 000
A multipart message with many MIME parts each containing many language tags
can exhaust memory: the per-part RFC 2231 parser had no cumulative limit.
Add remaining_language_tags to struct message_part_data_limits, initialised
to MESSAGE_PART_DATA_MAX_TOTAL_LANGUAGE_TAGS (100 000). Break out of the
tag-parsing loop in parse_content_language() once the budget reaches zero.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Gbp-Pq: Name 0013-lib-mail-Limit-total-Content-Language-tag-count-per-.patch
Timo Sirainen [Fri, 17 Apr 2026 13:55:48 +0000 (15:55 +0200)]
[PATCH 12/14] lib-mail: Limit total address count per message to 100 000
A message with millions of addresses across all its envelope headers can
exhaust memory when parsed (each struct message_address is ~100 bytes
regardless of whether the raw address is only a few bytes long).
Add remaining_addresses to struct message_part_data_limits, initialised to
MESSAGE_PART_DATA_MAX_TOTAL_ADDRESSES (100 000). Pass the remaining budget
as max_addresses to message_address_parse_full() so parsing stops at the
limit, then deduct the actual count parsed from the budget.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Gbp-Pq: Name 0012-lib-mail-Limit-total-address-count-per-message-to-10.patch
Timo Sirainen [Fri, 17 Apr 2026 13:54:48 +0000 (15:54 +0200)]
[PATCH 11/14] lib-mail: Introduce struct message_part_data_limits and thread it through parsers
Add an empty struct message_part_data_limits and pass it by pointer through
message_part_data_parse_from_header() and message_part_envelope_parse_from_header().
No behaviour change: the struct has no fields yet and no limits are applied.
Subsequent commits add the individual limit fields and enforcement.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Gbp-Pq: Name 0011-lib-mail-Introduce-struct-message_part_data_limits-a.patch
Timo Sirainen [Sun, 19 Apr 2026 21:16:14 +0000 (00:16 +0300)]
[PATCH 10/14] lib-mail: Clamp hdr->value cumulatively against header_block_max_size
Until now header_block_max_size only bounded hdr->full_value via
value_buf. Continued chunks returned to the caller via hdr->value were
left at the raw chunk size, so a caller that consumed hdr->value per
chunk without ever requesting use_full_value (e.g. the header-cache
path in index_mail_parse_header()) could accumulate the full raw header
size. A pathological To: with millions of addresses could grow
mail->header_data and the cache write buffer to tens of megabytes each,
driving the imap process over vsz_limit on FETCH ENVELOPE.
Reinterpret header_block_total_size as the running sum of line_value_size
across all chunks of all headers, and clamp each new chunk against the
remaining header_block_max_size budget. Propagate the clamped size to
line->value_len in the two continued-line branches that previously left
it untouched. value_buf is bounded implicitly since every append uses
line_value_size. The up-front per-chunk clamp
(line->value_len = MIN(value_len, max_size)) is now subsumed by the
cumulative clamp and has been removed.
Update the truncation tests that were documenting the old
"value_len stays at raw chunk size" behavior.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Gbp-Pq: Name 0010-lib-mail-Clamp-hdr-value-cumulatively-against-header.patch
Timo Sirainen [Sun, 19 Apr 2026 15:28:16 +0000 (18:28 +0300)]
[PATCH 09/14] lib-storage: Cap per-header size in index_mail_get_header_stream()
Apply MESSAGE_HEADER_BLOCK_DEFAULT_MAX_SIZE (10 MB) to the header-filter
istream used for populating the header cache. A pathological single
header (for example a To: with millions of addresses) otherwise grows
mail->header_data and the cache write buffer in lockstep with the raw
header size, which can push the imap process over vsz_limit on FETCH
ENVELOPE / BODYSTRUCTURE.
On its own this change does not yet bound hdr->value delivery; that
requires the upcoming change to message_parse_header_next() to clamp
per-chunk value_len cumulatively. Setting the limit here now lets that
follow-up take effect without further touching this file.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Gbp-Pq: Name 0009-lib-storage-Cap-per-header-size-in-index_mail_get_he.patch
Timo Sirainen [Sun, 19 Apr 2026 15:15:49 +0000 (18:15 +0300)]
[PATCH 08/14] lib-mail: Make istream-header-filter's per-header size limit configurable
Until now the internal message_header_parser_ctx used the default
MESSAGE_HEADER_BLOCK_DEFAULT_MAX_SIZE (10 MB) cap. In practice that cap
only bounded hdr->full_value; per-chunk hdr->value was delivered in full
regardless. A subsequent change to message_parse_header_next() will
clamp hdr->value cumulatively as well, and at that point the 10 MB
default would silently truncate data for callers like mbox-save that
stream raw header bytes to storage.
Default the filter to SIZE_MAX (unlimited) to preserve the effective
behavior and add i_stream_header_filter_set_max_header_block_size() so
callers that genuinely want a cap (index_mail_get_header_stream()) can
opt in.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Gbp-Pq: Name 0008-lib-mail-Make-istream-header-filter-s-per-header-siz.patch
Timo Sirainen [Sun, 19 Apr 2026 22:14:02 +0000 (01:14 +0300)]
[PATCH 07/14] lib-mail: istream-header-filter - Use container_of() for struct casts
Replace the C-style downcasts from struct istream_private/iostream_private
to struct header_filter_istream with container_of(), matching the
convention used by other istream implementations in tree.
Gbp-Pq: Name 0007-lib-mail-istream-header-filter-Use-container_of-for-.patch
Timo Sirainen [Sun, 19 Apr 2026 12:57:08 +0000 (15:57 +0300)]
[PATCH 06/14] lib-storage/mbox: Explicitly disable the header block size limit
mbox_sync_parse_next_mail() appends each header's raw bytes into
ctx->header and writes them back when rewriting the mbox file. The
default MESSAGE_HEADER_BLOCK_DEFAULT_MAX_SIZE limit (10 MB) must not
apply here: any truncation would corrupt the mbox on rewrite.
Similarly, mbox_sync_parse_match_mail() feeds full raw header bytes
into the MD5 verifier.
Today the per-chunk hdr->value delivered by message_parse_header_next()
is not clamped cumulatively, so the limit only bites on full_value and
unknown headers pass through intact. That is about to change - a
subsequent commit will clamp hdr->value cumulatively. Explicitly
setting SIZE_MAX here locks in the intent and prevents a regression.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Gbp-Pq: Name 0006-lib-storage-mbox-Explicitly-disable-the-header-block.patch
Timo Sirainen [Fri, 17 Apr 2026 13:46:10 +0000 (15:46 +0200)]
[PATCH 05/14] lib-mail: Add count field to struct message_address_list
Add a count field and increment it in add_address() so callers can read
the number of parsed addresses directly from the list struct without
walking the linked list.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Gbp-Pq: Name 0005-lib-mail-Add-count-field-to-struct-message_address_l.patch
The preparsed (from-parts) header parser was not tracking
all_headers_total_size and never called message_parse_header_lower_limit(),
so the cumulative 50 MB header size limit was never applied when re-parsing
a message using cached part structure.
Fix by mirroring the same tracking that message-parser.c does in its
parse_next_header_block(): update all_headers_total_size for each parsed
header line, and call message_parse_header_lower_limit() with the remaining
budget when initialising the per-part header parser.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Gbp-Pq: Name 0004-lib-mail-message-parser-from-parts-Enforce-50-MB-all.patch
Timo Sirainen [Fri, 17 Apr 2026 14:55:34 +0000 (16:55 +0200)]
[PATCH 03/14] imap: Stream ENVELOPE to client without ostream memory duplication
Same fix as for BODYSTRUCTURE: avoid copying the envelope string into
the ostream ring buffer by using o_stream_set_max_buffer_size(0) +
o_stream_send_istream() with a continuation handler.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Gbp-Pq: Name 0003-imap-Stream-ENVELOPE-to-client-without-ostream-memor.patch
Timo Sirainen [Fri, 17 Apr 2026 14:55:05 +0000 (16:55 +0200)]
[PATCH 02/14] imap: Stream BODYSTRUCTURE to client without ostream memory duplication
For huge BODYSTRUCTURE responses (messages with many MIME parts),
o_stream_send_str() caused the ostream to copy the entire string into
its ring buffer when the stream was corked, doubling memory usage.
Use the same o_stream_set_max_buffer_size(0) + o_stream_send_istream()
pattern as fetch_stream_continue() so the ostream buffer stays at 0
bytes and the I/O loop handles flow control via WAIT_OUTPUT.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Gbp-Pq: Name 0002-imap-Stream-BODYSTRUCTURE-to-client-without-ostream-.patch
Timo Sirainen [Sat, 28 Feb 2026 08:27:19 +0000 (10:27 +0200)]
[PATCH 01/14] lib-mail: Reset charset translation buffer between MIME parts
If MIME part ended with an incomplete charset translation, the buffer was
kept for the next MIME part. This could have produced garbage in the next
MIME part, or a crash.
[PATCH] lib-imap: imap-match - Fix excessive CPU usage caused by backtracking
Backport the NFA/non-backtracking algorithm from 2684624… while retaining the
byte-oriented matcher semantics of 2.4.1.
Original commit message:
Replace the recursive backtracking matcher with a Thompson-style NFA
simulation over grapheme clusters.
Each grapheme cluster of the compressed pattern becomes one state:
LITERAL, PERCENT (consume any number of non-separator clusters) or STAR
(consume any number of clusters including separators). A virtual
ACCEPT position sits at index n_states. Simulation tracks the set of
active positions in a bitmap. Epsilon-closure (skipping a PERCENT or
STAR without consuming) is a single forward pass because the NFA is
linear: each state can only epsilon-skip to i+1.
The resulting match is O(n_data * n_pattern) regardless of pattern
shape, with no recursion and no backtracking, so there is no way for
a malicious pattern or mailbox name to trigger exponential CPU,
excessive stack depth, or unbounded memory.
IMAP_MATCH_YES / NO / CHILDREN / PARENT semantics are preserved:
- YES: ACCEPT reachable after consuming all data.
- PARENT: ACCEPT was active at some point while the next data
grapheme cluster was the separator.
- CHILDREN: some active non-ACCEPT state remains after consuming all
data, and either the data ends with a separator or an
active state can still consume a separator (precomputed
as sep_accept[]).
Inboxcase handling (case-insensitive comparison for the INBOX prefix
of data) and grapheme-cluster comparison are unchanged - the existing
match_gc logic is reused inline as literal_matches().
pattern_compress() and pattern_is_inboxcase() are unchanged.
Gbp-Pq: Name 0001-lib-imap-imap-match-Fix-excessive-CPU-usage-caused-b.patch
Add a dedicated oauth2_audience setting checked against the token's aud
claim (RFC 7519 section 4.1.3) using the same tab-split AND-semantics as
oauth2_scope. Emit a deprecation warning when the existing aud fallback
in db_oauth2_token_in_scope() is triggered so operators know to migrate.
Gbp-Pq: Name 0003-auth-db-oauth2-Add-oauth2_audience-setting-deprecate.patch
Aki Tuomi [Fri, 29 May 2026 06:28:40 +0000 (09:28 +0300)]
[PATCH 6/6] auth: passdb_sql: connect before expanding query variables
sql_lookup_pass() calls settings_get_params() with an SQL escape func that
requires a live connection. If the DB is disconnected, the error propagated
as "Failed to parse configuration" instead of a DB connectivity error.
Call sql_connect() first; fail with "Not connected to database" directly
if it returns -1.
Gbp-Pq: Name 0006-auth-passdb_sql-connect-before-expanding-query-varia.patch
Aki Tuomi [Fri, 29 May 2026 11:22:57 +0000 (11:22 +0000)]
[PATCH 5/6] lib-sql: Make escape_string return int with error_r, fail instead of unsafe fallback
Change escape_string driver vfunc and sql_escape_string() to return int
with separate error_r output parameter. On failure (e.g. not connected),
return -1 instead of falling back to unsafe escaping.
Move escaping from sql_statement_bind_str() to sql_statement_get_query()
so errors can be propagated to callers. Add failed_error field to
sql_transaction_context for deferred error reporting at commit time.
Gbp-Pq: Name 0005-lib-sql-Make-escape_string-return-int-with-error_r-f.patch
Aki Tuomi [Thu, 21 May 2026 08:00:56 +0000 (11:00 +0300)]
[PATCH 4/6] lib-var-expand: Change escape func signature to return int with error_r
Wire error propagation in var_expand_program_execute_one_real() so a
failing escape function causes the expansion to return -1.
Update all implementations (passdb-sql, userdb-sql, db-ldap, dict-ldap,
auth-request-var-expand) and replace the unsafe auth_request_escape_func_t
cast in auth-request-var-expand.c with a proper bridge wrapper.
Gbp-Pq: Name 0004-lib-var-expand-Change-escape-func-signature-to-retur.patch
Aki Tuomi [Mon, 6 Oct 2025 09:39:50 +0000 (12:39 +0300)]
[PATCH] lib-sql: sql-api - Implement async calls for drivers that can't
Changes the behaviour of asynchronous functions to be truly asynchronous
even if the underlying driver isn't capable of doing this. Implemented
by adding immediate timeouts to call the callbacks after returning from the
synchronous function and ending up back to ioloop.
Gbp-Pq: Name 0001-lib-sql-sql-api-Implement-async-calls-for-drivers-th.patch
Aki Tuomi [Wed, 10 Jun 2026 11:54:00 +0000 (11:54 +0000)]
[PATCH 2/4] lib-compression: Iterate instead of recursing on zero-output decompress chunks
istream-lz4, istream-zlib and istream-bzlib each retried a read that
produced no output by tail-calling their own read function. A crafted
compressed stream can contain an unbounded number of chunks/steps that
each decompress to zero bytes - e.g. an lz4 stream of single-byte chunks
that each decode to nothing - so an attacker controlling the compressed
data can drive recursion depth proportional to the chunk count. Tail-call
optimization is not guaranteed, so this can exhaust the stack and crash
the process reading the stream.
Replace the recursive calls with continue inside the for(;;) loop
introduced in the previous commit, keeping stack usage O(1). Add a
regression test that feeds istream-lz4 a stream of 100000 empty chunks
and reads it in a single call.
Gbp-Pq: Name 0002-lib-compression-Iterate-instead-of-recursing-on-zero.patch
Aki Tuomi [Wed, 10 Jun 2026 11:53:39 +0000 (11:53 +0000)]
[PATCH 1/4] lib-compression: wrap *_read() in for(;;) loop (reindent only)
Prepare for the next commit: wrap the body of i_stream_lz4_read(),
i_stream_bzlib_read(), and i_stream_zlib_read() in a for(;;) loop.
No logic change; all paths still return on the first iteration.
Separating the indent churn makes the actual fix easier to review.
Gbp-Pq: Name 0001-lib-compression-wrap-_read-in-for-loop-reindent-only.patch
Aki Tuomi [Fri, 12 Jun 2026 11:19:42 +0000 (11:19 +0000)]
[PATCH 3/4] lib-compression: istream-zstd - Guard against no-progress loop in read
If ZSTD_decompressStream() returns with neither input consumed nor
output produced, the read loop would spin indefinitely. Add a check
after each call: if input.pos is unchanged and output.pos is zero,
treat it as a corrupt stream (EIO).
Gbp-Pq: Name 0003-lib-compression-istream-zstd-Guard-against-no-progre.patch
Aki Tuomi [Fri, 12 Jun 2026 11:19:42 +0000 (11:19 +0000)]
[PATCH 4/4] lib-compression: test - Add zero-len frame test for all handlers
Verify that successive empty compressed frames produce a clean EOF
across all compression algorithms. lz4 is limited to one frame as it
uses a custom single-stream format without concatenation support.
Gbp-Pq: Name 0004-lib-compression-test-Add-zero-len-frame-test-for-all.patch
Timo Sirainen [Mon, 4 May 2026 13:10:13 +0000 (13:10 +0000)]
[PATCH 2/2] lib-sieve: storage: file - Validate script stat path through dir_fd
Extend the symlink-escape protection added in the previous commit to the
stat performed by sieve_file_script_open(): an "include :personal" lookup
or any other indirect path that triggers sieve_file_script_stat() also
needs to refuse a symlink whose target leaves the personal storage
directory, otherwise the existence check succeeds and the file is opened
later via the safe path with an unhelpful "permission denied".
Add sieve_file_script_stat_safe(), which uses fstatat(AT_SYMLINK_NOFOLLOW)
to obtain the entry's own stat (lnk_st) and then, only if the entry is a
symlink, opens it through sieve_file_storage_open_safe() to validate the
target stays inside dir_fd and to fetch the resolved target's stat (st)
via fstat(). Non-symlink entries skip the open entirely.
Use the new helper in sieve_file_script_open() whenever fstorage->dir_fd
is available, falling back to the unsafe lstat+stat variant for
non-personal or single-file storages.
Gbp-Pq: Name 0002-lib-sieve-storage-file-Validate-script-stat-path-thr.patch
Pigeonhole's file storage followed any symlink encountered while resolving a
script path, including symlinks in personal (user-writable) storage whose
target lay outside the storage directory. In some non-recommended
configurations a user could exploit this through the include extension:
an "include :personal name;" lookup of ~/sieve/name.sieve transparently
followed a user-placed to e.g. another user's file readable by the mail
process, leaking its contents (or causing it to be parsed as Sieve).
Normally this shouldn't be possible, because sieve processes shouldn't
have any more privileges to read files than the local system user creating
the symlink.
Open the canonical (realpath'd) personal storage directory at storage init
time and keep an O_DIRECTORY|O_CLOEXEC fd to it. Resolve script content
reads through this fd by routing sieve_file_script_get_stream() via a new
sieve_file_storage_open_safe() wrapper around t_openat_safe(), which:
- opens each path component with O_NOFOLLOW so symlinks are detected
explicitly rather than transparently followed;
- follows symlinks only when their (recursively resolved) target stays
beneath dir_fd, refusing absolute targets and `..` past the storage
root with ELOOP;
- caps the symlink-hop count to bound resolution time.
Anchoring at dir_fd makes the lookup TOCTOU-safe even when intermediate
path components are mutated on disk concurrently: resolution stays
relative to the original directory inode and the safe walker still rejects
any target that leaves it.
Apply the protection only when storage->is_personal is set; admin-managed
global storage is trusted and may legitimately use cross-boundary symlinks.
Single-file storages (is_file=TRUE) keep dir_fd at -1 and fall back to the
existing open path. Also guard the dir_fd open with S_ISDIR() to handle the
autodetect quirk where storage_path can refer to a regular file even when
is_file is FALSE.
Gbp-Pq: Name 0001-lib-sieve-storage-file-Refuse-symlinks-escaping-pers.patch
Add a helper that opens a path relative to a directory fd while refusing
any resolution that escapes that directory. Each component is opened with
O_NOFOLLOW so symlinks are detected explicitly rather than transparently
followed; symlinks are then resolved manually and only honoured when their
(recursively resolved) target also stays beneath the base fd. Absolute
symlink targets, leading '/', and '..' past the base are rejected with
errno=ELOOP. Symlink chains are bounded by
PATH_UTIL_OPENAT_SAFE_MAX_SYMLINKS hops.
Anchoring resolution at a caller-held base_fd makes the lookup TOCTOU-safe
even when intermediate path components are mutated on disk concurrently:
the kernel resolves all openat() lookups relative to the original
directory inode that base_fd refers to, so an attacker mutating path
components on the filesystem cannot redirect resolution.
This is the portable counterpart to Linux's openat2() with RESOLVE_BENEATH
and is intended for callers that need to safely look up files under a
user-writable directory (e.g. a Sieve script storage directory).
Gbp-Pq: Name 0002-lib-path-util-Add-t_openat_safe-for-jailed-openat-re.patch
Timo Sirainen [Thu, 11 Jun 2026 19:33:23 +0000 (19:33 +0000)]
[PATCH 2/2] lib-sieve: actions - Create fileinto mailbox without the autocreate flag
Stop setting MAILBOX_FLAG_AUTO_CREATE/SUBSCRIBE in act_store. Instead, on
mailbox_open() returning MAIL_ERROR_NOTFOUND, create the mailbox explicitly
with sieve_act_store_create_mailbox() when autocreation is enabled (the
:create side effect already creates it explicitly).
This makes the creation go through the normal ACL CREATE gate, so a fileinto
(with lda_mailbox_autocreate=yes) or fileinto :create can no longer create a
mailbox in a shared or public namespace that the user lacks the CREATE right
for.
A permission failure to create the mailbox is classified by
sieve_act_store_create_error_status(): for a user's personal script it is a
permanent failure (implicit keep), while for a global (administrator-defined)
script it is deferred (temporary failure), so the administrator notices the
script is behaving in a way they did not intend.
Gbp-Pq: Name 0002-lib-sieve-actions-Create-fileinto-mailbox-without-th.patch
Timo Sirainen [Thu, 11 Jun 2026 19:31:45 +0000 (19:31 +0000)]
[PATCH 1/2] lib-sieve: actions - Split mailbox creation into sieve_act_store_create_mailbox()
Pure refactoring with no functional change: move the mailbox_create() /
subscribe / reopen sequence out of the "mailbox" extension's :create side
effect (seff_mailbox_create_pre_execute) into a new reusable
sieve_act_store_create_mailbox() helper.
Gbp-Pq: Name 0001-lib-sieve-actions-Split-mailbox-creation-into-sieve_.patch
Timo Sirainen [Thu, 11 Jun 2026 17:20:40 +0000 (17:20 +0000)]
[PATCH 2/2] lib-storage: Remove MAILBOX_FLAG_AUTO_CREATE and MAILBOX_FLAG_AUTO_SUBSCRIBE
These flags were set only by mail delivery (lib-lda) and Sieve fileinto
(Pigeonhole), both of which now create the target mailbox explicitly with
mailbox_create(). Nothing sets the flags anymore, so remove them and drop
the flag checks from mailbox_is_autocreated() and mailbox_is_autosubscribed().
As a result mailbox_is_autocreated() reflects only mailboxes that virtually
already exist (the user's INBOX and admin-configured "auto" mailboxes), so
the ACL plugin again enforces the CREATE right for delivery-autocreated
mailboxes in shared and public namespaces.
Gbp-Pq: Name 0002-lib-storage-Remove-MAILBOX_FLAG_AUTO_CREATE-and-MAIL.patch
Instead of relying on MAILBOX_FLAG_AUTO_CREATE to have mailbox_open()
implicitly create the target, open it and, on MAIL_ERROR_NOTFOUND,
create it explicitly when lda_mailbox_autocreate is enabled. This makes
the creation go through the normal mailbox_create() path (including ACL
checks) and removes one of the two users of the autocreate mailbox flags.
Gbp-Pq: Name 0001-lib-lda-mail-deliver-Autocreate-delivery-mailbox-via.patch
Timo Sirainen [Sat, 4 Jul 2026 16:04:04 +0000 (16:04 +0000)]
[PATCH] imap-login: Fix excessive memory growth with pre-login ID command
The pre-login ID parser reset the per-argument line and list limits after
every argument, so nothing bounded the number of key/value pairs. Each
external key grew log_reply and added a distinct event field (a linear
scan, making it O(N^2)), allowing a pre-auth CPU/memory amplification.
Account at most 30 pairs (per RFC 2971) into the logging and client_id
bookkeeping. Internal x-* parameter handlers still run for pairs beyond
the limit, since a trusted proxy may legitimately forward more fields.
Gbp-Pq: Name 0001-imap-login-Fix-excessive-memory-growth-with-pre-logi.patch
Aki Tuomi [Mon, 25 May 2026 14:54:03 +0000 (14:54 +0000)]
[PATCH] lib-sieve: Reduce CPU limit check interval in :contains and :matches matchers
Reduce the interval to 64 so worst-case work between checks is ~64 MiB.
cpu_limit_exceeded() is a signal-counter comparison with no syscall, so
the higher call frequency has no measurable cost.
Gbp-Pq: Name 0008-lib-sieve-Reduce-CPU-limit-check-interval-in-contain.patch
Timo Sirainen [Wed, 3 Jun 2026 12:25:49 +0000 (12:25 +0000)]
[PATCH 7/7] lib-sieve: Log an info line when opening a CPU-disabled script
sieve_binary_check_executable() blocks execution when a script's
persisted cumulative CPU usage already exceeds sieve_max_cpu_time, but
only logged the block at debug level. With cumulative tracking now
sticky across redeliveries via the per-user sieve-rusage file, the
script can stay blocked indefinitely until the resource_usage_timeout
decay or admin intervention; operators benefit from a per-attempt
record of that block at info level.
Promote the existing debug log to e_info and include the cumulative
CPU usage. The caller continues to emit its own error message on the
failed open (lda-sieve, imap-sieve, imap-filter-sieve plugin paths).
Gbp-Pq: Name 0007-lib-sieve-Log-an-info-line-when-opening-a-CPU-disabl.patch
Timo Sirainen [Wed, 3 Jun 2026 12:25:38 +0000 (12:25 +0000)]
[PATCH 6/7] lib-sieve: Log a warning when cumulative CPU exceeds the limit
sieve_binary_record_resource_usage() returned FALSE when the script's
cumulative CPU time crossed sieve_max_cpu_time, but the only log line
on that path was a debug-level "Updated cumulative resource usage"
emitted on every recording, with no signal at the actual disable
transition. Operators had to infer the disable from later "Failed to
open script ... (cumulative resource limit exceeded)" errors on
subsequent deliveries.
Emit a warning at the transition point, including the cumulative CPU
usage that triggered it.
Gbp-Pq: Name 0006-lib-sieve-Log-a-warning-when-cumulative-CPU-exceeds-.patch
Timo Sirainen [Tue, 5 May 2026 16:51:27 +0000 (16:51 +0000)]
[PATCH 5/7] lib-sieve: file storage: Remove orphan compiled binaries on delete and rename
DELETESCRIPT and RENAMESCRIPT only operated on the .sieve file and
left the matching compiled binary (.svbin) on disk. The orphan was
never referenced again under that name, so it accumulated as wasted
disk space until something else (manual cleanup, account removal)
removed it.
Unlink fscript->bin_path alongside the .sieve operation. The unlink is
best-effort: failures other than ENOENT are logged but do not fail the
DELETE/RENAME.
Gbp-Pq: Name 0005-lib-sieve-file-storage-Remove-orphan-compiled-binari.patch
Timo Sirainen [Tue, 5 May 2026 16:30:23 +0000 (16:30 +0000)]
[PATCH 4/7] lib-sieve: Switch sieve_max_cpu_time enforcement to time-decayed re-enable
When a user's cumulative CPU time exceeded sieve_max_cpu_time, the
script was permanently disabled. SIEVE_BINARY_FLAG_RESOURCE_LIMIT was
set in the binary header on first overrun and was never cleared by
the resource_usage_timeout decay (which only zeroed the cpu_time
field), so sieve_binary_check_executable() refused execution forever
unless the binary was recompiled or an admin manually wiped the file.
Replace the sticky-flag model with one driven purely by the persisted
cumulative cpu_time, decayed by resource_usage_timeout:
- Drop SIEVE_BINARY_FLAG_RESOURCE_LIMIT and the enum that defined it;
nothing reads sbin->header.flags anymore.
- sieve_binary_check_executable() and sieve_binary_check_resource_usage()
call sieve_resource_usage_is_excessive() on the current cumulative
cpu_time. Raising sieve_max_cpu_time now correctly re-enables a
previously over-limit user without further intervention.
- Gate per-user file persistence on the per-run delta, not the
cumulative total. A refused-without-running attempt has a zero
delta, skips the file write, and leaves update_time at the moment
work was actually consumed. After resource_usage_timeout seconds
of inactivity (or only-trivial deliveries) the next load decays
the cpu_time to zero and the user is allowed to run again.
The on-disk per-user sieve-rusage file format becomes:
V1 <cpu-secs> <update-time>
Gbp-Pq: Name 0004-lib-sieve-Switch-sieve_max_cpu_time-enforcement-to-t.patch
Timo Sirainen [Wed, 3 Jun 2026 12:10:46 +0000 (12:10 +0000)]
[PATCH 3/7] lib-sieve: Rename on-disk header rusage fields to unused_*
The .svbin header's resource_usage fields no longer carry data: they
are zeroed on save and discarded on load. Cumulative CPU tracking
lives in the per-user sieve-rusage file, and the in-memory carry-over
lives in sbin->persisted_rusage.
Flatten the nested resource_usage struct and rename the fields to
unused_update_time and unused_cpu_time_msecs so it is obvious the
fields are kept only for on-disk format compatibility. The memory
layout of struct sieve_binary_header is unchanged.
Gbp-Pq: Name 0003-lib-sieve-Rename-on-disk-header-rusage-fields-to-unu.patch
Timo Sirainen [Tue, 5 May 2026 16:19:20 +0000 (16:19 +0000)]
[PATCH 2/7] lib-sieve: Drop binary-header resource usage tracking
With the per-user sieve-rusage file now authoritative for personal
storages, the .svbin header no longer needs to carry CPU usage. Admin
storages (sieve_before/after, default location) cannot be re-uploaded
by the user, so cross-run tracking is unnecessary there; the runtime
sieve_max_cpu_time check still enforces the per-execution limit.
Move the in-memory cumulative carry-over from sbin->header.resource_usage
to a dedicated sbin->persisted_rusage substruct. The on-disk header
struct now solely represents the wire format and is zeroed on every
save; the in-memory carry-over no longer overloads it. Drop the now-dead
"resource usage" section from sieve_binary_dumper_run() (the on-disk
fields are always zero, so the section never produced output).
Gbp-Pq: Name 0002-lib-sieve-Drop-binary-header-resource-usage-tracking.patch
Timo Sirainen [Tue, 5 May 2026 16:18:34 +0000 (16:18 +0000)]
[PATCH 1/7] lib-sieve: Add per-user sieve-rusage file for cumulative CPU tracking
Cumulative resource usage was previously stored in each compiled Sieve
binary's (.svbin) header. Because the binary file is keyed by script
name, a user could reset their CPU budget by uploading the same script
under a different name (PUTSCRIPT newname + SETACTIVE newname),
RENAMESCRIPT, or DELETESCRIPT followed by PUTSCRIPT under another
name. Each such operation produced a fresh .svbin with zero counter,
allowing the sieve_max_cpu_time limit to be bypassed indefinitely.
Track cumulative CPU per user instead, in <INBOX-namespace-root>/sieve-rusage.
The file persists across all script renames, deletions, and re-uploads,
since it is not tied to any particular script name. Tracking is enabled
only for personal storages where the user's INBOX namespace has a
filesystem path; admin/default storages are unaffected here.
The on-disk format is a single ASCII line:
V1 <flags> <cpu-secs> <update-time>
CPU time is rounded up to whole seconds when persisted; sub-second
precision is preserved only for the in-memory accumulation within a
single execution. The file is updated by writing a new copy via
file_dotlock_open_mode() and replacing the original via
file_dotlock_replace(). Concurrent writers are serialized through the
dotlock; readers do not need a lock because the rename is atomic.
The .svbin header still carries its original resource_usage struct as
the active fallback when no per-user file is available; a follow-up
commit drops that path.
Gbp-Pq: Name 0001-lib-sieve-Add-per-user-sieve-rusage-file-for-cumulat.patch
Stephan Bosch [Wed, 4 Mar 2026 20:53:36 +0000 (21:53 +0100)]
[PATCH] lib-sieve: sieve-storage - Fix panic caused by flawed detection of personal storage type
Recent commit d82c2e624e4a9c364a6fa58fad0938ae5578660c added the is_personal
flag, but it is assigned too late. Fix this issue by assigning this flag in the
earliest allocation function.
This issue commonly surfaces when people forget to set an appropriate type for a
script storage definition, in which case the "personal" type is the default.
Stephan Bosch [Tue, 25 Nov 2025 04:07:16 +0000 (05:07 +0100)]
[PATCH] lib-sieve: sieve-storage - Add is_personal bit
Simpler than evaluating the type and personal storage is created only through
sieve_storage_create_personal(). This mimics the bit for default storage.
Gbp-Pq: Name 0001-lib-sieve-sieve-storage-Add-is_personal-bit.patch
Timo Sirainen [Tue, 23 Jun 2026 15:26:51 +0000 (15:26 +0000)]
[PATCH 2/2] lib-storage: Simplify mailbox_mkdir() using mailbox_list_mkdir_root(NULL)
mailbox_mkdir() looked up the root path via mailbox_list_get_root_forced()
only to pass it straight to mailbox_list_mkdir_root(). Let mkdir_root()
resolve the root itself by passing path=NULL, dropping the local variable.
Gbp-Pq: Name 0002-lib-storage-Simplify-mailbox_mkdir-using-mailbox_lis.patch
Timo Sirainen [Tue, 23 Jun 2026 14:59:45 +0000 (14:59 +0000)]
[PATCH 1/2] lib-storage: Allow NULL path in mailbox_list_mkdir_root()
When path is NULL, look up the root directory path of the given type via
mailbox_list_get_root_path() and create that. If there is no path of the
requested type, nothing is done. This avoids open-coding the common
get_root_path() + mkdir_root() pair at call sites.
Gbp-Pq: Name 0001-lib-storage-Allow-NULL-path-in-mailbox_list_mkdir_ro.patch
Timo Sirainen [Mon, 6 Jul 2026 07:17:42 +0000 (07:17 +0000)]
[PATCH 2/3] doveadm: compress-connect - Don't hang when server rejects COMPRESS
After sending a COMPRESS command the client stopped reading stdin and
waited for the server's reply, but it only resumed on an "OK Begin
compression" reply. If the server rejected COMPRESS (e.g. a NO reply for
an unsupported mechanism), compress_waiting was never cleared: the client
waited forever for an OK that never came while the server waited for the
next command, deadlocking the session.
Remember the COMPRESS command tag and treat any tagged reply to it that
isn't the compression-start reply as a rejection, resuming the pipelined
input.
Gbp-Pq: Name 0002-doveadm-compress-connect-Don-t-hang-when-server-reje.patch
Timo Sirainen [Sat, 4 Jul 2026 21:57:15 +0000 (21:57 +0000)]
[PATCH 1/3] imap: Restrict COMPRESS to DEFLATE only
cmd_compress() and the imap-login proxy side channel looked up any
registered compression handler by name, so a client could request e.g.
COMPRESS ZSTD even though only COMPRESS=DEFLATE is advertised (RFC 4978).
Non-DEFLATE algorithms can require far more memory to decompress (a zstd
decoder alone can hold a 16+ MB window per connection versus ~40 KB for
inflate). This matters now that imap-login handles COMPRESS for proxied
connections: an attacker opening many connections could drive the shared
login process into its vsz_limit and crash it, affecting all clients on
that process.
Only accept DEFLATE on both the backend and the login proxy path.
Gbp-Pq: Name 0001-imap-Restrict-COMPRESS-to-DEFLATE-only.patch
Timo Sirainen [Wed, 22 Apr 2026 12:44:24 +0000 (15:44 +0300)]
[PATCH 2/3] imap-acl: Fail if ACL identifier is invalid
Reject invalid identifiers early in imap_acl_identifier_parse() using
acl_id_is_valid(). This prevents CR/LF injection and rejects identifiers
that are too long, contain control characters or are not valid UTF-8.
Timo Sirainen [Sun, 19 Apr 2026 23:10:29 +0000 (23:10 +0000)]
[PATCH] lib-sieve: Enforce CPU time limit within :contains and :matches matcher loops
The naive O(N*M) substring search in mcht-contains.c and the naive find loop
in mcht-matches.c can run for hours on a large value (e.g. a message body),
completely bypassing sieve_max_cpu_time because that limit was only checked
between bytecode operations.
Expose the active CPU limit via sieve_runtime_cpu_limit_exceeded() and poll
it every 4096 inner iterations. When the limit is hit the match returns
SIEVE_EXEC_RESOURCE_LIMIT, matching the existing behavior at the bytecode
boundary. This is a minimal safety net ahead of switching the matchers to
algorithms that do not require it.